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Abstract 

Image segmentation is to separate an image into distinct 
homogeneous regions belonging to different objects. It is 
an essential step in image analysis and computer vision. 
This paper compares some segmentation technologies and 
attempts to find an automated way to better determine the 
parameters for image segmentation, especially the connec- 
tivity value of X in X-connected segmentation. 

Based on the theories of the maximum entropy 
method and Otsu's minimum variance method, we pro- 
pose:( 1 )maximum entropy connectedness determination: a 
method that uses maximum entropy to determine the best X 
value in X-connected segmentation, and (2) minimum vari- 
ance connectedness determination: a method that uses the 
principle of minimum variance to determine X value. Apply- 
ing these optimization techniques in real images, the experi- 
mental results have shown great promise in the development 
of the new methods. In the end, we extend the above method 
to more general case in order to compare it with the famous 
Mumford-Shah method that uses variational principle and 
geometric measure. 



1 Introduction 



split-and-merge segmentation, to name a few. A-connected 
segmentation is a technique in the category of region grow- 
ing segmentation. It was proposed to find an object having 
the property of gradual variation 01 H (TOl. 

This paper attempts to find an automated way to bet- 
ter determine the parameters for image segmentation, espe- 
cially the connectivity value of A in A-connected segmenta- 
tion. 

This paper first reviews some major segmentation tech- 
niques to explain why segmentation is difficult, and how 
a special technique would be selected in specific applica- 
tions. We then focus on our problem of determining seg- 
mentation parameters in A-connected segmentation. Based 
on the philosophies of the maximum entropy method and 
Otsu's minimum variance method, we propose: (^maxi- 
mum entropy connectedness determination: a method that 
uses maximum entropy to determine A value, and (2) mini- 
mum variance connectedness determination: a method that 
uses the principle of minimum variance to determine A 
value. Applying these optimization techniques in real im- 
ages, the experimental results have shown the great promise 
of the new methods. In the end, we extend the above method 
to a more general case in order to compare it with the fa- 
mous Mumford-Shah method that uses variational principle 
and geometric measure lf20l . 



Image segmentation is the basic approach in image pro- 
cessing and computer vision 11221 . It is used to locate special 
regions and then extract information from them. Image seg- 
mentation is used to partition an image into different com- 
ponents or objects and is an essential procedure for image 
preprocessing, object detection and extraction, and object 
tracking. Image segmentation is also related to edge detec- 
tion. 

Even though there is no unified theory for image seg- 
mentation , some practical methods have been studied over 
the years such as thresholding, edge based segmentation, re- 
gion growing, clustering (unsupervised classification), and 



2 Image Segmentation Review 

In this section, we first review currect technology of im- 
age segmentation: five types of techniques, their character- 
istics, and uses. We then focus on the connectedness-based 
image segmentation technique. 

2.1 Overview of Image Segmentation Ap- 
proaches 

As we know, there is no unified theory for image seg- 
mentation, some practical methods have been studied over 
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the years such as thresholding, edge based segmentation, 
region growing, clustering (unsupervised classification, e.g. 
k-mean or fuzzy c-mean), and split-and-merge segmenta- 
tion. These segmentation algorithms have been developed 
for solving different problems ||26ll . However, they are all 
based on one or more of the five philosophies listed below: 

(1) A segment is a class/cluster, so one can use a clas- 
sification/clustering method to segment the image. Clas- 
sification methods usually do not need to use the loca- 
tion/position information. Clustering for unsupervised clas- 
sification technology can perform better to find an object 
for sampled points within the subset of data frames. Typi- 
cal techniques include Isodata and k-mean or fuzzy c-mean. 

if -mean or fuzzy c-mean is a standard classification 
method that is often used in image segmentation [24]. This 
method classifies the pixels into different groups in order to 
minimize the total "errors," where the "error" is the distance 
from the pixel value to the center of its own group. 

(2) A segment is a homogeneous region. If an object 
or region can be identified by absolute intensity (the pixel 
value), we usually use threshold segmentation. In other 
words, an object will be recognized as a geometrically con- 
nected region whose values/intensities are between a certain 
high-limit and a low-limit. We usually assume that the high 
limit is the highest value of the image. Therefore, in prac- 
tice, one only needs to determine the low-limit. Maximum 
entropy and minimum variance (also called Otsu's method) 
are two of the most popular methods for determining the 
best threshold for single image H). 

Multilevel thresholding is similar to threshold segmenta- 
tion and uses the same philosophy, but multiple thresholds 
are produced at once. It needs an extremely high time cost 
for computation [22| [29 1 . We will discuss these two meth- 
ods in detail in section 3. 

(3) A segment is a "smoothly-connected" region. In a 
region where intensity changes smoothly or gradually, the 
region is viewed as a segment. Smoothness can be mea- 
sured by a limit. 

A popular segmentation method is called mean-based re- 
gion growing segmentation in this paper fl3l . A pixel will 
be included in a region if the updated region is homoge- 
neous, meaning that the difference between pixel intensity 
and the mean of the region is limited by e, a small real num- 
ber. 

The A-connected segmentation follows the same philos- 
ophy. This is to link all pixels that have the similar intensity. 

This method is related to a fuzzy method created by 
Rosenfeld who treated an image as a 2D fuzzy set. Then, 
he used ce-cut to segment the image into components. An- 
other way is to measure two pixels to see if they are "fuzzy" 
connected. A pixel set is A-connected if for any two points 
there is a path that is A-connected where A is a fuzzy value 
between and 1 HUH E3E1- This is a generaliza- 



tion of threshold segmentation in some cases. This method 
can be used to divide (partition) different intensity levels 
without calculating different thresholds or clip-level values. 
However, for a complex image, how to calculate the value 
of A remains unknown. 

A fast algorithm can be designed to perform a segmenta- 
tion. In fact, the simple form of both threshold segmentation 
and A-connected segmentation can be done in linear time. 

We can see that mean-based region growing segmenta- 
tion is a statistical approach, but the A-connected approach 
is a graph-theoretic method. We can combine them by re- 
quiring that the A-connected segment also be within a e limit 
of the mean. Or after the mean-based segmentation, we can 
do a A-connected segmentation. 

(4) Split-and-merge segmentation uses quadtree to deter- 
mine the order in which pixel(s) should be treated or com- 
puted [ 23 1 . It is an algorithmic way to find an object or to 
force a merge order. This is because this method is based 
on the mean of the merged region. It does not guarrantee 
a transitive relation. Again, the mean-based segmentation 
is not an equivalence relation. This method splits an im- 
age into four sections and checks if each part is homoge- 
nous. The homogenous segments are then merged together. 
If the segments are not homogenous, the splitting process 
is repeated. This process is also called quadtree segmenta- 
tion. The method is more accurate for some complex im- 
ages. However, it costs more time to segment an image. 
The time complexity of this method (process) is 0(nlogn). 
This was proved by Chen in 1991.[4| 

(5) A segment is surrounded by one or several closed 
edges. If we can detect and track the edges, we can deter- 
mine the location and outline of the segments. 

The fifth philosophy is edge detection. To find low or 
high frequency pixels are very common in edge detection. 
However, not all edge-detection methods can be used in im- 
age segmentation since enhancing edge is not the primary 
purpose of image segmentation. The purpose of image seg- 
mentation is to find components. The number of edges 
should be relatively small. Otherwise, the extraction of the 
closed curves will be the major problem. Recent develop- 
ment indicated that the Mumford-Shah method is promis- 
ing. The method uses the variational principal [20|. This 
method has captured a considerable amount of attention. 

The Mumford-Shah method considers three factors in 
segmentation: (1) the total length of all the segments?? 
edges, (2) the unevenness of the image without its edges, 
and (3) the total error between the original image and 
the proposed segmented images where each segment has 
unique or similar values in its pixels. When the three 
weighted factors are minimized, the resulting image is a 
solution of the Mumford-Shah method. Recently, Zhan 
and Vese proposed level-sets to simplify the Mumford-Shah 
method so that it produced better results ifTTl . However, 
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level-sets use contour boundaries that may limit the flexibil- 
ity of the original Mumford-Shah method. The Mumford- 
Shah method also needs an alternative process and its algo- 
rithm performance is still unknown. 

2.2 Remarks on Selecting an Appropriate 
Segmentation Method 

The fc-mean or fuzzy c-mean, maximum entropy, and the 
Mumford-Shah method all require an iterated process that 
is good at detailed or fine segmentation. This is not a quick 
solution for fast segmentation. The fast segmentation meth- 
ods are only used for region growing including the orig- 
inal threshold method, mean-based and lambda-connected 
search, and split-and-merge method. 

In practice, fc-mean or fuzzy c-mean and maximum en- 
tropy are still the most popular. However, for some types 
of sequential images, these two methods do not obtain sta- 
ble results when we process a set of meteorological data to 
find areas with the most water vapor indicating (most likely) 
the location of a hurricane fl9l - A-connected segmentation, 
on the other hand, has worked very well. A problem the 
A-connected method faced was that the A value needed to 
be assigned even though that value can be used throughout 
each data frame in the data set. To find a way to automati- 
cally determine the A-value is a longtime goal for the author. 
Some methods have been proposed such as maximum con- 
nectedness spanning tree and the golden cut point 
0. 

In this paper, the author proposes optimization methods 
based on maximum entropy and minimum variance, respec- 
tively. 

2.3 The Connectedness-based Segmenta- 
tion 

A-Connected segmentation is based on the philosophy 
that an object must Have a smooth inside and has a gap at 
its boundary (in terms of intensity). Trying to find a closed 
curve/boundary that indicates the intensity of the jump is 
the key to this method. 

A measure of connectedness can be used to partition a 
set of data into connected components based on adjacency 
or neighborhood systems. Using connectedness to divide an 
image does not require transforming the image into binary 
form. After the data is partitioned, a fast algorithm such as 
the breadth-first-search algorithm can be applied to find a 
connected component [3|[8|. A-connected search was in- 
troduced to segment such an image without transferring the 
image into a {0, l}-image. However, the value of A, usually 
between and 1, determines the fineness of the segmenta- 
tion. 



A-connectedness can be defined on an undirected graph 
G = (V, E) with an associated (potential) function / : 
V — > R rn , where R m is the m-dimensional real space ifTUll . 
Given a measure a p (x,y) on each pair of adjacent points 
x, y based on the values p(x), p(y), we define 

i \ ( p(p(x), p(v)) if x and y are adjacent ... 
a p (x,y = <^ rv\ J'HWJJ y J (1) 

py ' \Q otherwise 

where fx : R m x R m — » [0,1] with (i(u,v) — (J,(v,u) 
and p(u,u) = 1. a p is used to measure "neighbor- 
connectivity." The next step is to develop path-connectivity 
so that A-connectedness on < G, p > can be defined in a 
general way. 

In graph theory, a finite sequence xx, X2, ■-, x n is called 
a path, if (2^, ar<+i) G E. The path-connectivity j3 of a path 

7r = n(xi, x n ) — {x\, X2, Xn) is defined as 

/3 p (ir(xi,x n )) = min{a p (xi,x i+ i)\i = 1, ...,n - 1}(2) 

or 

Pp(7r{xi,x n )) = Y\_{a P (x l ,x l+ i)\i = 1, ...,n- 1}(3) 

Finally, the degree of connectedness or connectivity of two 
vertices x, y with respect to p is defined as: 

C p (x, y) — max{/?(7r(a;, y))|7r is a (simple) path.} (4) 

For a given A G [0,1], point p — (x,p(x)) and q = 
(y,p(y)) are said to be A-connected if C p (x,y) > A. In 
image processing, p(x) is the intensity of a point x and 
p = (x, p{x)) defines a pixel. 

3 A Value Determination and Optimization 

It is a natural and unavoidable question how we deter- 
mine A value in connectedness-based segmentation? It is 
somehow similar to determine the clip level in threshold 
segmentation; however, lambda value is not as sensitive as 
the clip-level in thresholding. There are fewer A values to 
be selected than clip-levels. In Chapter 10 of [6|, Chen pro- 
vided the detail analysis on this issue. 

Some techniques have been proposed and tested such as 
the binary search-based method and the maximum connect- 
edness spanning tree method J6] 0. We also proposed a 
golden-cut technique for bone density measurement in Q. 
In this section, we propose two new methods to determine 
the A value for the segmentation. The new methods are 
based on maximum entropy and minimum variance, respec- 
tively. 

3.1 Connectedness and Maximum En- 
tropy 

In this subsection, we propose a method that uses the 
maximum entropy method to determine the A value. It can 
be called maximum entropy connectedness determination. 
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The maximum entropy method was first proposed by Ka- 
pur, Sahoo, and Wong [15] • It is based on the maximization 
of inner entropy in both the foreground and background. 
The purpose of finding the best threshold is to make both 
objects in the foreground and background, respectively, as 
smooth as possible. H3 |22) [l] 

If F and B are in the foreground and background classes, 
respectively, the maximum entropy can be calculated as fol- 
lows; 

where pi can be viewed as the number of pixels whose 
value is i; p(B) is the number of pixels in background, 
and p(F) is the number of pixels in foreground. The maxi- 
mum entropy is to find the threshold value t that maximizes 
H F {t)+H B {t). 

Such an idea can be used for A-connected segmentation. 
However, the total inner entropy for the image is to calculate 
the entropy for each segment (A-connected component), not 
for the thresholding clipped foreground/background. This 
is because in A-connected segmentation there is no specific 
background. Each A-connected segment can be viewed as 
foreground, and the rest may be viewed as the background. 
It is different from the original maximum entropy where 
the range of pixel values determines the inclusion of pixels. 
Therefore, we need to summarize all inner entropies in all 
segments. 



H{\) = S™ xi?i(A)) (6) 

The maximum entropy connectedness can be viewed as 
a measure of a special connectivity for the image. If A value 
is calculated in the above formula for an image that makes 
H (A) to be maximum, we call that the image have the max- 
imum entropy connectedness A, denoted as A e . 

3.2 Experimental results with A e 

In 0, we proposed a golden cut method for finding the 
A-value for bone density connectedness calculation. We 
have obtained a A=0.96, 0.97 for a bone image (the size of 
the picture is different from the one used in this paper). For 
a similar image, using the maximum entropy connectedness 
presented in this section, we got A e =0.95. The result is quite 
reasonable. The original image and both of the segmented 
images are shown in Fig. 1-4. No pre-cut (preprocessing) is 
performed in the segmentation. 




Figure 1. Bone Density Image Segmentation 
: the Original image 



H (A) = S(inner entropy of each A-connected component) 

(5) 

We will select the A such that H(X) will be maximized. 
We call this A-value the maximum entropy connectedness. 
This unique value is a new measure for images. 

Since the maximum entropy means the minimum 
amount of information or minimum variation, we want the 
minimum change inside each segment. This matches the 
philosophy of the original maximum entropy method. In 
other words, the A-connected maximum entropy has a bet- 
ter meaning in some applications. We use the A e such that 

H{X e ) = man{H{x)\x £ [0, 1]}. 

Assume there are m A-components, define inner entropy 
of each A-component Si'. 

IT /\^ ^255 Histogram[k] Histogram[k] 
HiiX) = Sfe =° n l0g n 

where n is the number of points in the component Si. 
Histogram[k] is the number of pixels whose values are k 
in the segment. Thus, 




Figure 2. Bone Density Image Segmentation 
: A=0.97 

What we can see in the above three segmented images 
(Fig. 2-4). Fig. 2 seemed to be the same as the original 
image Fig. 1. It is possible that Fig. 4 may represent the 
better understanding of bone connectivity. However, what 
we state here is that A e can provide us the meaningful result 
and it was done automatically. 

For the commonly used testing image "Lena," Fig. 5, 
the result of A-connected segmentation is quite interesting. 
Whatever we use a pre-threshold cut or not, A e is always 
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Figure 3. Bone Density Image Segmentation 

: A e =0.95 




Figure 4. Bone Density Image Segmentation 
: A=0.93 



0.99 (Fig. 6). An original maximum entropy arrived at the 
clip-level of 125 counts of the 8 -bit gray level image (0- 
255 of the pixel value range. The reason is that the "Lena" 
image does not contain many "continuous" parts. In the 
A-connected segmentation, we can see that A e (=0.99) con- 
nected segmentation has connected the continuous compo- 
nent especially at the face and shoulder. This matches the 
result of using the maximum entropy cut, Fig. 7 (We use 
NIH ImageJ to perform the cut.) Thus, we can say that 
our new method is still reasonable. When we use A = 0.98 
for the image, we get Fig. 8. 




Figure 5. Image Segmentation for testing im- 
age "Lena:" the Original image 




Figure 6. Image Segmentation for testing im- 
age "Lena:" A=0.99 




Figure 7. Image Segmentation for testing im- 
age "Lena:" Standard Maximum Extropy 



For the image having gradual variation property, A- 
connected segmentation usually has an advantage. We have 
extensively tested a set of sequential images in order to 
find the outlier of meteorological data that indicates (most 
likely) the hurricane center fl9l . The data frames we used 
are water vapor images. 

Except the standard threshold method and the A- 
connected segmentation method, all other methods we 
tested failed including famous fc-mean and maximum en- 
tropy |19|. What we found was that the key for this se- 
quential images is that the pre-cut is necessary. Interesting 
enough for us, 45% of the pick value of each image for the 
cut receives the best result. After that, we can use A=0.95 
for the segmentation parameter and extract the largest com- 
ponent for the outlier searching result. 

The problem here is that 45% of the pick value as clip- 
level is not automatically calculated. If we want a totally au- 
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Figure 8. Image Segmentation for testing im 
age "Lena:" A=0.98 



tomatic process, we might to use maximum entropy to make 
the first cut. In this testing set, we have 12 image frames. 
For some beginning images, the method worked well as ex- 
pected. For other images, the new method proposed in this 
paper consistently got wrong results since the A e calculated 
is always greater than or equal to 0.97. We cannot get A 
to be 0.95 using "maximum entropy connectedness." After 
looking into the detail images, we have found two prob- 
lems: (1) the pixel values are not "continuous" around "the 
desired outlier," and (2) The largest component criteria for 
the outlier is not quite represent the nature of hurricane cen- 
ters (outliers), we need to change the criteria to "the largest 
and the brightest." 

For (1), we have done a smoothing process. For (2), we 
change the outlier criteria from the largest component to the 
total intensity of the component (not only testing its size). 
Smoothing preprocess is reasonable, and it is good for the 
A-connected segmentation to find large component. 

The following image shows such a treatment. We have 
applied an automatic pre-cut by using maximum entropy 
instead of the pre-cut using the 45% of pick value. Fig. 9 
shows an original image. 

Fig. 10 shows the result using our new method plus a 
maximum entropy threshold cut of preprocessing. A thresh- 
old value=23 is calculated by standard maximum entropy. 
Then, we perform the automated finding process to get A e 
described above. A e = 0.90 was obtained and used. 

If we just use the standard maximum entropy at threshold 
= 23, we will have the following image Fig. 11. 

3.3 Consider Outer Entropy in Maximum 
Entropy Connectedness 

Using maximum entropy is a type of philosophical 
change. In fact, we can consider other formulas. For ex- 




Figure 9. Image Segmentation for Meteoro- 
logical data outlier: Original Image 



ample we can select another way to calculate the entropy of 
one segment. 

Here we propose a different formula. We can calculate 
the inner entropy of a component, then treat the rest of the 
image as the background for the component. The total en- 
tropy generated by this segment is the summation of both. 
We can apply this process to all components/segments while 
segmenting. 

Let / be the image, Cj(A) = I—Si(X) is the complement 
of component 5,(A) 

H(d(X)) = {Entropy for the set d(X)} 

We can use the following formula for the the basis of opti- 
mzation. 

H(X) = H(Si(X)) + H(d(X)) 
The outer (background) entropy is the total. 

H(outer) = T,H(Ci) 

In H(outer), a pixel is calculated multiple times. We may 
need to use the average H (outer) / m where m is the num- 
ber of segments. The relationship between this formula and 
the formula we used in the previous subsection is also inter- 
esting. 

Furthermore, we should consider the following general 
model. 

H optimal = a ■ H (inner) + b ■ H (outer) 
where a and b can be constant or function of segmentations. 
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Figure 10. Image Segmentation for Meteoro- 
logical data outlier: A e = 0.90, maximum en- 
tropy connectedness determination. The im- 
age is smoothed and pre-cut by the standard 
maximum entropy at threshold = 23. 



Figure 11. Image Segmentation for Meteoro- 
logical data outlier: The image is smoothed 
and clipped by standard maximum entropy at 
threshold = 23. 



3.4 Connectedness and Minimum (Inner) 
Variance 

In this subsection, we develop a minimum variance- 
based method for finding the best A value in A-connected 
segmentation. Minimum variance was first studied by Otsu 
in image segmentation [2JJ [TT]. In other words, Otsu's seg- 
mentation was the first global optimization solution for im- 
age segmentation. It is used to clip the image into two parts: 
the object and the background. 

Assume that a 2 (W),a 2 (B), a 2 (T) represent the within- 
class variance, between-class variance, and the total vari- 
ance, respectively. The optimum threshold will be deter- 
mined by maximizing one of the following criterion with 
respect to threshold t [21] [TT] : 

° 2 {B) a 2 {B) a 2 (T) 
a 2 (W) ' cr 2 (T) ' <T 2 (W) 

a is the standard derivation. Since cr 2 (T) is constant 
for a certain image, this segmentation process is to make 
between-class variance large and within-class variance 
small. Therefore, our task is to make the within-class vari- 
ance as small as possible. 

The original design of Otsu's method is not able to be 
implemented directly for A-connected segmentation which 
is similar to the case of maximum entropy. This is because 
there were only two categories, the object and the back- 
ground. For the first and second criteria, in A-connected 
segmentation there are many components and it would be 
very hard to find between-class variance. We could con- 
sider the total between-class variance, by considering every 



pair of components. Or we could consider between-class 
variance for the components that are the neighbors . The 
third criterion seems likely to be valid, however, when we 
only consider the variance within a connected component, 



- 1 gig) 



will be 



what will happen is a 2 (W) = if A = 1. a 2(w) 
infinite and will always be the greatest value. 

In order to make use of Otsu's philosophy, we modify 
the original formula by adding a term that is the number 
of segments or components, M, We try to minimize the 
following formula: 

H(\) = S(inner variance of eachA — component) + c • M 

(7) 

where c is a constant. We could let c = 1. The calculation 
of the inner variance of a A-component is to compute the 
variance (square of standard deviation) of the pixels in the 
component. 

The following formula is to find minimum average vari- 
ance (for each component). 

H (A) = S(inner variance of eachA — component) /M 

(8) 



We want to find A,, such that 



H(X V ) = min{iJ(A)|A £ [0,1]} 



(9) 



This strategy only works for the meteorological data. 
The experimental results show that the method is promis- 
ing. For the other two kinds of images tested in maximum 
entropy connectedness, "Lena" and the Bone image, we still 
need to find an appropriate way under minimum variance 
philosophy. 
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The following images show the process on the same pic- 
ture with a preprocessing threshold cut using the maximum 
entropy cut or 45% peak cut. Then, we perform the auto- 
mated process of finding of A-value. Fig. 12 shows that we 
arrived at A,j=0.97 using the method of minimum variance 
connectedness determination described in this subsection. 
Without smoothing the original image, we pre cut the im- 
age using maximum entropy threshold. The result is not 
what we expected. 




Figure 12. Minimum variance connectedness 
determination without smoothing 



When we smoothed the image, we got A t) =0.90, and the 
result turned to be correct, see Fig. 13. 




Figure 13. Minimum variance connectedness 
determination with smoothing 



3.5 A-Connectedness and Mumford- 
Shah's Method 

How do we use Mumford-Shah's idea to find the optimal 
segmentation? We can define L as the total length of the 
edge of all segments. 

H (A) = a ■ E(inner variance of each A-component) + j3 ■ L 

(10) 

where a and j3 are constants. More generally, we can do the 
normal A-connected fit |4| [9 1 on each A-connected compo- 
nent. The total variance (or standard deviation) of the (nor- 
mal A-connected) fitted image is denoted as V . L is still 
the total length of edges of segments (components), and D 
is the difference between the fitted image and the original 
image. Using Mumford-Shah's Method, we can minimize 
the following equation to get the X p . 

H(X) = a-(V) + (3-(L) + j-(D) (11) 
H(X p ) = mm{H{\)\\ € [0, 1]} (12) 

4 Discussion 

Even though we calculated the entropy or variance in 
each connected component that is different from the stan- 
dard maximum entropy and the Otsu's method in image seg- 
mentation, the philosophy remains the same as in these two 
popular methods. The results are very promising. These 
two new methods can be easily applied in other region- 
growing segmentations. A large amount of further research 
should be done to support and the new methods. We will 
implement the method proposed in subsection E in section 
III, and compare it with the results obtained in ifTTl . 
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